Groundbreaking Simultaneous and Heterogeneous Multi-Threading Technology to Make Computing Faster

Although all new devices from tech giants like Apple and Google feature incremental improvements—a single-digit increase in battery life, one less nanometer for the processor, which is yet to produce optimal yield for the manufacturers, or a few extra megapixels—the question arises: Are such modest enhancements truly enough? Is adding more hardware the solution?

Not according to Associate Professor Hung-Wei Tseng from the Department of Electrical and Computer Engineering at the University of California, Riverside (UCR). He says: 

“You don’t need to add new processors because you already have them.”

Professor Tseng, along with a team of researchers, developed a new software framework for parallel processing called Simultaneous and Heterogeneous Multi-threading (SHMT). According to initial results, SHMT is poised to significantly enhance processing speed and reduce power consumption by tapping into the latent capabilities of current processors in personal computers, cellphones, and other devices.

Touted as “ground-breaking” by the tech community, SHMT aims to remove data flow bottlenecks and facilitate the seamless collaboration of many processing units. This breakthrough may affect not just personal electronics but also data centers and other kinds of massively parallel computing.

Click here to learn how advanced photonics will enable us to build better smartphones.

Breaking Down the Bottleneck

Before we set out to explore the full glory of what is achievable with simultaneous and heterogeneous multi-threading, let us first understand the limitations of current computing systems. 

In most devices, various components, like the central processing unit (CPU), graphics processing unit (GPU), and tensor processing unit (TPU), handle information separately. Data is transferred from one processing unit to another, often resulting in “bottlenecks” that hinder overall system performance.

This is further aggravated by the traditional programming models, which typically delegate tasks to a single type of processor, thereby leaving other resources idle and underutilized. Echoing these observations, the research paper ‘Simultaneous and Heterogeneous Multi-threading’ by Kuan-Chieh Hsu and Hung-Wei Tseng states: 

“The entrenched programming models focus on using only the most efficient processing units for each code region, underutilizing the processing power within heterogeneous computers.”

SHMT takes a deviation from this approach by exploiting the diversity of multiple components within a computing system. This concept is known as heterogeneity. By breaking down computational functions and distributing them among available processing units, SHMT facilitates true parallel processing. 

This approach of decomposing computational functions and distributing them among several processing units maximizes the utilization of available resources to improve performance and save energy. The research paper further dissects the shortcomings of traditional programming models by stating that they “can only delegate a code region exclusively to one kind of processor, leaving other computing resources idle without contributing to the current function.” 

SHMT, on the other hand, aims to break free from these constraints by leveraging each processing unit’s distinct skills and their collaborative work on a shared code region. The authors also point out that contemporary computing technology is undeniably heterogeneous, as all computing platforms integrate multiple types of processing units and hardware accelerators. This calls for a programming model that can effectively harness the power of these diverse components (which is exactly what SHMT aims to achieve).

Hence, SHMT paves the way for faster and more efficient computing by addressing the bottlenecks in now-traditional computing. 

How do Simultaneous and Heterogeneous Multi-Threading Technology Work?

As evident, managing and distributing computing activities efficiently among different hardware components is the basic principle behind SHMT. 

The framework includes a collection of virtual operations (VOPs) to facilitate the offloading of tasks from a CPU application to a virtual hardware device. According to the study, “A set of virtual operations (VOPs) allows a CPU program to ‘offload’ a function to a virtual hardware device.” These VOPs mediate communication and job delegation by creating a barrier between the program and the hardware.

A runtime system optimizes performance by evaluating the capabilities of each hardware resource and making intelligent scheduling decisions while the application is being executed. According to the study, “During program execution, a runtime system drives simultaneous and heterogeneous multi-threading’s virtual hardware, gauging the hardware resource’s ability to make scheduling decisions.” To maximize resource efficiency and adapt to job-specific needs, SHMT dynamically evaluates hardware capabilities.

The runtime system breaks down VOPs into high-level operations (HLOPs) to distribute them to various hardware task queues. According to the study, “The runtime system divides VOPs into one or more high-level operations (HLOPs) to simultaneously use multiple hardware resources.” Decomposing VOPs into HLOPs achieves granular control over job allocation and maximum utilization of each processing unit.

The SHMT scheduling policy is based on quality-aware work-stealing (QAWS), which helps to keep resources used efficiently and workload varied. According to the study, “SHMT uses a quality-aware work-stealing (QAWS) scheduling policy that doesn’t hog resources, but helps maintain quality control and workload balance.” In addition to distributing work effectively across the system, this approach stops any processing unit from hoarding resources.

If SHMT wants to maximize performance without sacrificing quality, it needs the QAWS scheduling policy. The study states that “SHMT must assure the outcome without incurring significant overhead.” To guarantee that the output from heterogeneous processing units is accurate and consistent, SHMT integrates quality control techniques into scheduling.

The ability of SHMT to make use of the specific capabilities of each piece of hardware is a major plus. As the study notes, “SHMT can break up the computation from the same function to multiple types of computing resources and exploits heterogeneous types of parallelism in the meantime.” SHMT greatly enhances performance because it makes use of parallelism in heterogeneous systems to run jobs simultaneously across several processor units.

Another aspect of SHMT that is supposed to be flexible and adaptive is the runtime system. And according to the study, “As HLOPs are hardware-independent, the runtime system can adjust the task assignment as required.” Because of its adaptability, SHMT can react on the fly to changes in hardware availability or workload demands, keeping the system running at peak efficiency and performance.

Overall, the study lays out all the necessary steps for understanding how SHMT operates, drawing attention to the critical parts and processes that allow it to accomplish remarkable efficiency and effectiveness in heterogeneous computing environments. Thanks to SHMT, which uses VOPs, HLOPs, and the QAWS scheduling strategy to revolutionize parallel processing, a new age of efficient and powerful computing is about to dawn.

Positive Findings from Initial Testing of the Prototype

To prove that SHMT works, the researchers at UCR ran rigorous tests on a prototype system that mimicked data center capabilities by employing parts standard in contemporary cellphones. The prototype included a Google Edge TPU incorporated via the system’s M.2 Key E slot, an NVIDIA Jetson Nano module with a quad-core ARM Cortex-A57 processor, and 128 Maxwell architecture GPU cores.

To evaluate the SHMT framework’s performance under different workload circumstances, the researchers ran the prototype through a battery of benchmark programs. The outcome was impressive: the top-performing QAWS strategy not only reduced energy usage by 51% but also enhanced processing performance by 1.95X compared to the baseline technique.

The results underscore SHMT’s potential to greatly improve processing performance and energy efficiency across a broad spectrum of devices and software applications. It demonstrated that it is possible to get the most out of your current setup by making better use of all the resources it already has without having to spend a fortune on new hardware.

With the ever-increasing need for faster and more efficient computing, breakthroughs such as simultaneous and heterogeneous multi-threading will become increasingly crucial in shaping the future trajectory of technology. The work of the UCR research team makes it clear that finding long-term, high-performance computing solutions capable of adapting to the dynamic demands of our digital world has never been easier than with the work of the UCR research team.

Implications and Future Directions of Simultaneous and Heterogeneous Multi-Threading

The creation and testing of SHMT represent a profound shift in the future of computing. It has the potential to revolutionize computing device design and usage across several applications by offering substantial performance increases and energy savings with existing hardware.

As SHMT gains wider adoption, consumers may be able to avoid expensive hardware updates and enjoy quicker, more responsive mobile devices, tablets, laptops, and desktops. Because of this, more people will soon be able to purchase and have access to high-performance computers, helping close the digital divide.

Data centers and other large-scale computing systems, too, might find SHMT to be an indispensable tool for cutting costs and energy usage without sacrificing performance. Additionally, innovations that promote energy efficiency and sustainability, such as SHMT, will gain significance as worries about technology’s environmental effects escalate.

Despite their best efforts, the UCR research team recognizes that there are still obstacles to be conquered and opportunities for more investigation and advancement in the future. Software engineers and hardware makers will need to work closely together to implement SHMT on a large scale. This will guarantee that the technology works well on all devices and platforms. However, further research is required to determine which applications and workloads are most suited to using this revolutionary technology.

Notwithstanding these obstacles, academics and businesses alike have taken notice of SHMT’s promising early results. The possibility that this ground-breaking technology may transform the computer industry is becoming increasingly attractive as studies progress and collaborations are established. 

Like many other brilliant ideas, simultaneous and heterogeneous multi-threading seems to be a product of common sense, but the devil is in the details. While the idea of a shared cache between CPUs and GPUs is intriguing, it is likely to require a complete overhaul of the hardware architecture.

It would warrant moving away from the current x86-64 architecture, and such a design would necessitate the development of a new processor architecture with a shared L3 or L4 cache. This would, in turn, increase the complexity of the CPU and potentially negate any benefits gained from the shared cache. 

On top of that, cache memory is typically much smaller compared to system RAM and is not well-suited for GPU applications, which require large amounts of high-bandwidth memory. However, developments like universal memory may address these concerns. As research into SHMT continues, it will be exciting to see how this innovative technology evolves and impacts the future of parallel processing and heterogeneous computing.